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Improvements  in  helmet  mounted  eye  gaze  point  sensor  technology  are  needed  for 
aircrew  performance  assessment  in  both  simulation  and  in-flight  testing.  The  task 
reported  in  this  Technical  Memorandum  is  a  design  and  proof-of-concept  effort 
for  development  of  a  low-cost  oculoraeter  to  meet  this  need.  The  work  was 
conducted  for  NAVAIRSYSCOM  under  AIRTASK  A9303D/053E/7W1812009 ,  Work  Unit 
NATC-3044.  Subsequent  design  refinement  is  planned;  however,  this  report 
completes  the  requirements  of  the  AIRTASK/Work  Unit. 
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SUMMARY 


This  report  documents  Phase  I  of  a  Small  Business  Innovation  Research  (SBIR) 
contract  for  development  of  a  low-cost  helmet  mounted  eye  gaze  point  sensor. 
The  device,  in  completed  form,  is  for  use  in  laboratory,  simulator,  and 
in-flight  studies  by  NAVAIRTESTCEN  in  aircraft  test  and  evaluation  projects. 
Numerous  other  potential  applications  in  behavioral  research  and  development 
would  benefit  from  use  of  the  device  throughout  the  Department  of  Defense  and  in 
commercial  and  academic  settings.  The  device  produces  a  time  history  of  eye  gaze 
point  information  along  with  pupil  diameter  and  eye-blink  data.  It  has 
flexibility  as  a  sensor  system  for  many  different  modes  of  application.  Sentient 
Systems  Technology,  Incorporated,  completed  design,  testing,  prototype  hardware 
and  software  development,  and  functional  testing  of  a  prototype  unit.  The  effort 
constituted  a  full  feasibility  demonstration  in  the  multistage  SBIR  contract 
cycle  format.  Success  in  Phase  I  is  intended  to  lead  to  follow-on  contracts  for 
engineering  development  and  applications  development  or  production  during 
Phases  II  and  III.  Functional  demonstrations  of  the  prototype  unit  with  Sun 
Microsystems  computer  equipment  were  convincing  and  fully  successful.  A  complete 
prototype  with  simplified  software  for  operation  with  an  IBM  compatible  PC 
system  is  now  being  evaluated  for  in-house  applications  in  a  behavioral  test  and 
evaluation  laboratory  setting.  The  Phase  I  contract  was  a  6  month  effort 
completed  in  March  1988.  The  full  text,  in  contractor  format,  of  Sentient 
Systems'  Final  Report  is  included  in  appendix  A.  Sentient  Systems  has  advanced  a 
Phase  II  proposal  (along  with  indications  of  Phase  III  and  commercial 
applications  interest)  through  the  Navy  approval  cycle  and  funding  for  the 
Phase  II  contract  is  now  pending.  Conclusions  are  that  the  technical  approach 
can  fully  meet  the  cost,  performance,  and  application  goals  of  the  effort. 
Recommendations  are  for  continuation  of  this  SBIR  effort  in  accordance  with  the 
Sentient  Systems  Technology,  Incorporated,  Phase  II  proposal. 
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INTRODUCTION 

1.  The  Aircrew  Systems  Department  of  the  Systems  Engineering  Test  Directorate, 

NAVAIRTESTCEN,  initiated  and  guided  a  test  and  evaluation  technology  development 
project  to  obtain  eye  gaze  point  monitoring  devices.  Sentient  Systems  Technology 
won  a  Small  Business  Innovation  Research  (SBIR)  competition  and  received  a 
Phase  I  SBIR  contract  in  1987  for  the  development  of  a  low-cost  oculometer. 

Requirements  for  the  device  included  that  it  be  low-cost,  lightweight, 
self-calibrating,  and  flightworthy.  The  Phase  I  Final  Report  by  Sentient  Systems 
Technology  provides  a  detailed  description  of  their  Phase  I  effort  and  is 
presented  in  appendix  A. 

2.  Some  important  points  of  the  Department  of  Defense  (DOD)  SBIR  program  are: 

a.  Goals  are  to  stimulate  technical  innovation  in  the  private  sector,  to 

enable  small  business  involvement  in  DOD  research  and  development,  and 
to  increase  commercial  application  of  DOD-supported  research  and 
development  results. 

b.  Agencies  of  the  DOD  identify  areas  where  research  and  development  effort 
is  desired.  Selected  task  descriptions  become  SBIR  contract  topics  for 
which  small  businesses  can  compete. 

c.  The  SBIR  program  consists  of  three  phases.  In  Phase  I,  6  month  contracts 

are  awarded  to  small  businesses  to  determine  the  technical  merit  and 
feasibility  of  a  proposed  concept.  Phase  II  awards  are  made  to  those 
companies  who  show  promise  in  Phase  I.  This  is  a  research  and 

development  effort  which  takes  no  more  than  2  years.  Phase  III  should 

involve  support  from  a  third  party  or  the  private  sector  and  lead  to 
production  or  application. 

RESULTS  AND  DISCUSSION 

3.  Phase  I  of  this  SBIR  project  for  development  of  a  low-cost  helmet  mounted 
oculometer  was  completed  in  March  1988  and  proved  feasibility  of  the  technical 
concept.  Results  included  a  working  prototype  model  which  is  self-calibrating, 
lightweight,  and  extremely  low-cost  but  still  works  with  the  required  accuracy. 
Its  outputs  include  precise  measurements  of  eye  movement,  blinks,  and  pupil 
size.  Sentient  Systems  Technology  has  completed  all  stated  goals  of  the  SBIR 
Phase  I  effort  and  has  completed  a  specific  account  of  the  work  in  the  Final 
Report  (appendix  A). 

4.  Sentient  Systems  Technology  provided  a  full  demonstration  of  the  prototype 
helmet  mounted  oculometer  to  the  NAVAIRTESTCEN  sponsors.  The  system  was 
controlled  by  a  Sun  computer  workstation  and  video  outputs  of  the  eye  and  pupil 
were  monitored.  The  prototype  system  functioned  as  intended  in  all  respects. 

5.  A  Phase  II  SBIR  proposal  was  received  and  approved  for  action  pending 
availability  of  SBIR  funds.  An  important  part  of  the  Phase  II  process  will  be 
iterative  NAVAIRTESTCEN  participation  in  testing  of  the  apparatus  between 
modifications  as  they  occur.  Three  separate  stages  of  design  refinement  and 
application  testing  are  planned.  Sentient  Systems  Technology  will  enhance  the 
prototype  hardware  for  convenient  use,  such  as  in  all  lighting  conditions  and 
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with  night  vision  goggles,  and  also  for  f lightworthiness.  Three  models  will  be 
made  for  different  flight  and  workload  conditions.  The  optics  will  also  be  made 
modular  for  use  with  any  helmet.  Refinement  of  the  prototype  software  will  occur 
and  will  permit  eventual  integration  of  the  oculometer  with  a  helmet  tracker  and 
3-D  computer  modeling  tools.  Application  products  using  the  system's 
computerized  outputs  are,  or  will  be,  gaze  point  time  history,  cockpit  and  field 
of  view  scoring,  crew  performance  data,  pupillometry  derived  scores,  and  task 
analysis  verification. 

6.  Strong  commercial  third  party  interests  in  this  product  have  been  indicated 
for  Phase  III  of  the  SBIR  program. 


CONCLUSIONS 

7.  Sentient  Systems  Technology  performed  all  of  the  tasks  called  for  in  the 
Phase  I  contract; 

a.  The  working  prototype  demonstrated  feasibility  of  the  design  concept  and 
that  it  can  meet  performance  and  cost  objectives.  The  suitability  of  the 
technical  approach  to  the  goals  of  this  development  effort  was  fully 
established. 

b.  Use  of  the  working  prototype  demonstrated  that  Sentient  Systems 
Technology  has  command  of  the  required  technology  and  permitted 
component  placement  tests  which  will  be  the  basis  for  the  detail  design 
of  a  working  system. 

c.  This  project  is  an  exemplary  model  of  the  intended  operation  of  the  SBIR 
program. 


RECOMMENDATION 

8.  This  effort  to  develop  a  low-cost  eye  gaze  point  sensor  should  be  continued 
in  accordance  with  SBIR  program  procedures  and  Sentient  Systems  Technology's 
Phase  II  proposal. 
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Project  Summary 

During  phase  I,  Sentient  Systems  Technolcgy  (SST)  has 
demonstrated  to  Navy  personnel  that  application  of  its  low 
cost,  eye  gaze  sensing  technology  can  result  in  a  practical, 
low-cost,  helmet  mounted  eye  gaze  sensing  system  for  use 
throughout  the  Department  of  Defense  in  human  factors, 
aerospace  medicine,  and  ultimately  operational  environments. 

SST  has  constructed,  and  demonstrated  to  Navy  personnel,  a  low 
cost,  helmet  mounted,  eye  gaze  sensing  testbed.  The  user  wears 
a  flight  helmet  to  which  a  lightweight,  compact,  CCD  camera  has 
been  attached.  The  camera  is  connected  to  proprietary  eye  image 
processing  electronics  which  allow  the  user's  point  of  gaze  and 
pupil  diameter  to  be  computed.  NATC  personnel  have  used  the 
working  phase  I  system  and  were  extremely  pleased  with  its 
operation.  The  testbed  has  allowed  prediction  of  the  ultimate 
price  and  performance  of  a  production  quality  device  should  it 
be  fully  developed  in  a  phase  II  effort. 
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Background  and  Significance  of  Problem 

In  human  factors  research,  there  is  a  tremendous  need  for 
determination  of  an  individual's  gaze  point.  Learning  to 
perceptually  select  those  visual  cues  that  are  most  informative 
concerning  particular  combat  tasks  are  an  important  part  of 
learning  to  deal  with  time  critical  visual  workloads.  For 
example,  in  flight  during  a  roll  turn,  the  visual  environment 
is  unusual:  the  ground  surface  is  seen  in  near  vertical 
perspective  with  the  oncoming  terrain  toward  the  top  of  the 
field  of  view  and  the  visual  cues  and  features  that  provide 
major  control  inputs  for  initiating,  maintaining,  modifying, 
and  completing  maneuvers  must  be  extracted  in  conditions  of 
high  stress.  Rapidly  extracting  critical  information  from 
instrumentation  and  from  the  external  combat  environment  may  be 
a  matter  of  life  or  death.  Proper  design  of  instrumentation, 
based  on  extensive  visual  scanning  behavior  research  and 
simulation,  could  result  in  dramatically  improved,  operational 
combat  effectiveness  for  the  aircraft  of  intent. 

In  addition,  it  is  apparent  from  the  effort  being  expended  by 
all  branches  of  the  armed  forces  on  eye  gaze  control  that  this 
technology  will  move  from  facilitation  of  simulation  displays 
to  control  of  operational  hardware.  The  report  "Trends  Shaping 
Advanced  Aircrew  Training  Capabilities  Through  The  1990's" 
written  for  the  Air  Force  Human  Resources  Laboratory  states: 

"In  short,  the  assumption  here  is  that  head  and  eye  coupling 
represents  a  critical  technology,  the  applications  of  which, 
will  become  increasingly  pervasive  throughout  the  decade  of  the 
nineties,  not  only  with  respect  to  simulator  display  technology 
but  also  with  respect  to  actual  in-flight  applications" 

( AFHRL-TP-84-52 ) .  One  motivation  for  this  development  is  that 
the  increasing  demands  on  personnel  for  vehicle  and  ordinance 
control  make  it  desirable  to  supplement  the  use  of  hands  and 
feet,  especially  for  secondary  tasks. 

Much  work  being  done  in  Department  of  Defense  medical  research 
labs  on  visual  behavior  physiology  would  also  benefit  from  the 
use  of  the  low  cost,  helmet  mounted  eye  gaze  sensor.  Monitoring 
point-of-gaze  and  pupil  diameter  can  tell  us  a  great  deal  about 
the  fundamental  performance  limitations  of  human  physiology.  As 
the  capabilities  of  our  machines  continue  to  increase,  fully 
understanding  the  limitations  imposed  by  their  human  operators 
will  become  as  important  as  understanding  the  laws  of  physics 
governing  the  design  of  these  advanced  military  systems. 

The  ultimate  cost  of  a  deliverable,  production  quantity  system 
is  one  of  the  primary  driving  forces  motivating  this  project. 

In  the  past,  eye  gaze  monitoring  systems  were  built  as 
one-of-a-kind,  high  cost  ( $50 , 000-$100 , 000 )  laboratory 
instrumentation  systems  and,  as  a  result,  the  potential  of  the 
technology  developed  for  them  was  never  fulfilled.  Because  SST 
manufactures  low  cost,  eye  gaze  controlled  systems  for  the 
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price-sensitive  commercial  market,  it  approaches  the 
development  of  eye  gaze  monitoring  systems  from  a  very 
different  perspective.  SST  and  NATC  personnel  feel  that  there 
is  a  large  military  and  commercial  market  for  a  low  cost,  high 
performance,  eye  gaze  sensing  device  that  would  allow  the 
potential  of  this  exciting  technology  to  finally  be  fully 
realized.  NATC  personnel  have  indicated  that  the  production 
quantity  price  must  be  under  $10,000.  Because  of  (a)  the  large 
production  volumes  of  compact,  lightweight,  high  resolution, 

CCD  cameras  and  their  resulting  low  cost  (under  $500);  and  (b) 
the  phase  I  demonstration  that  the  eye  image  processing 
hardware  for  a  single  camera  can  be  built  on  a  single  circuit 
board;  the  production  quantity  cost  constraint  can  be  easily 
attained. 
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Technical  Objectives  of  Phase  1  Work 

The  primary  objective  of  the  phase  I  effort  was  to  determine  if 
it  was  feasible  to  use  the  low  cost,  eye  gaze  sensing 
technology  developed  for  SST's  commercial  EyeTyper  product  line 
in  a  low  cost,  helmet  mounted  eye  gaze  sensing  system.  In 
addition  to  simply  proving  feasibility,  it  was  desired  to 
predict  the  ultimate  performance  and  price  of  such  a  low  cost, 
helmet-mounted,  eye  gaze  sensing  system  should  it  be  fully 
developed  in  a  phase  II  effort. 

The  desired  operational  parameters  for  the  low  cost,  helmet 
mounted  eye  gaze  device  were  identified  in  discussions  between 
SST  and  Navy  personnel  during  this  phase  I  effort.  They  are 
listed  below. 


1)  Added  helmet  weight  must  be  minimized  and  the  center  of 
gravity  should  not  be  changed;  If  the  COG  is  changed  it 
should  be  shifted  low  and  aft. 

2)  Obstruction  to  the  field  of  view  must  be  minimized. 

3)  Must  be  capable  of  being  used  with  night  vision  systems. 

4)  Production  quantity  devices  must  be  deliverable  for  less 
than  $10,000  per  device. 

5)  Gaze  resolution  should  be  a  minimum  of  2-3  degrees  with  a 
horizontal  range  of  plus  or  minus  20  degress  within  the  helmet. 

6)  Gaze  monitoring  speed  should  be  maximized. 

7)  It  is  desirable  to  monitor  and  count  blinks. 

8)  Calibration  should  be  easy,  fast,  and  robust. 

9)  Must  be  easily  interfaced  to  other  Navy  human  factors  computer 
systems.  Digital  output  should  be  provided  with  time 
synchronization . 

10)  A  measure  of  pupil  diameter  at  least  once  a  second. 

11)  Packaging  should  be  rugged  enough  to  demonstrate  that  flight 
worthiness  could  be  attained  with  further  development. 

12)  The  helmet-mounted  camera  packaging  should  provide  for  easy 
attachment  to,  and  detachment  from,  multiple  helmets. 


Because  combat  aircraft  normally  undergo  high  rates  of 
acceleration,  there  is  a  real  need  to  minimize  the  helmet 
weight.  In  simulator  and  human  factors  applications,  g-loading 
is  not  a  problem  but  the  helmet  weight  must  be  kept  realistic. 
An  ultimate  limit  on  operational  helmet  weight  is  related  to 
crash  survivability.  A  helmet-mounted  visual  display,  together 
with  the  eye  position  sensing  system,  must  not  significantly 
increase  the  weight  of  the  helmet.  Increasing  the  size  of  the 
helmet  is  also  unacceptable  in  the  limited  area  of  the  cockpit. 
In  military  aircraft,  the  primary  function  of  the  flight  helmet 
is  protection.  Since  this  is  not  the  case  in  simulators,  there 
are  fewer  constraints  on  mounting  additional  hardware  or 
modifying  the  helmet  in  order  to  reduce  the  weight. 
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If  the  eye  position  sensor  does  add  any  significant  weight  to 
the  helmet,  it  would  be  most  desirable  to  mount  the  system  as 
close  to  the  center  of  gravity  of  the  head/helmet  as  possible. 
If  this  is  not  done,  there  is  a  danger  of  rotational  torques 
induced  by  head  motion  causing  the  helmet  to  rotate  relative  to 
the  pilot's  head.  For  these  same  reasons,  any  cabling  attached 
to  the  helmet  must  not  increase  the  weight,  inertia,  and 
rotational  torque  induced  by  head  motion. 

Another  design  specification  for  this  system  is  the  number  of 
distinct  points  that  need  to  be  resolved  within  this  total 
field  of  view,  i.e.,  the  gaze  resolution.  The  minimum  number  of 
points  required  will  vary  widely  from  application  to 
application  however,  all  applications  undoubtedly  desire  as 
large  a  number  of  points,  resulting  in  the  highest  eye  gaze 
point  resolution,  as  is  technically  possible.  NATC  personnel 
have  indicated  that  a  minimum  gaze  resolution  of  2-3  degrees  is 
acceptable  with  a  40  degree  total  horizontal  range. 

A  major  constraint  in  the  design  of  a  helmet  mounted  eye 
position  sensor  is  that  the  distance  from  the  front  of  the  eye 
to  a  helmet  mounted  display,  helmet  visor,  night  vision 
goggles,  or  other  visual  enhancement/protection  devices  is  very 
small.  This  "eye  relief"  distance  is  typically  about  39  mm. 
This  makes  fabrication  of  the  eye  imaging  optics  one  of  the 
more  important  problems  to  be  surmounted. 

In  addition  to  eye  gaze  location  monitoring,  it  may  be 
desirable  in  many  applications  to  also  monitor  pupil  dilation. 
Pupil  dilation  may  be  an  essential  component  of  measuring 
workload  and  emotional  stress,  especially  when  combined  with 
other  workload/stress  measuring  techniques. 
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Phase  I  Work  and  Results 

We  will  now  describe  the  phase  I,  technical  feasibility, 
development  effort  which  resulted  in  the  construction  and 
evaluation  of  a  low  cost,  helmet  mounted  eye  gaze  sensing 
testbed . 

Initially,  the  desired  price-performance  goals  of  a  fully 
developed  system  needed  to  be  determined  in  discussions  with 
Navy  personnel.  These  discussions  resulted  in  a  detailed 
specifications  list  that  was  described  previously  in  this 
report.  SST  personnel  then  began  designing  and  building  a  low 
cost,  helmet  mounted  eye  gaze  sensing  testbed  which  would 
allow:  1)  demonstration  of  the  technical  feasibility  of  using 
technology  developed  for  SST's  EyeTyper  products  in  a  helmet 
mounted  system;  and  2)  exploration  of  the  helmet  mounted  eye 
gaze  sensing  system  design  space  to  allow  prediction  of  the 
ultimate  performance  and  price  of  such  a  low  cost, 
helmet-mounted,  eye  gaze  sensing  system  should  it  be  fully 
developed  in  a  phase  II  effort. 

All  eye  gaze  sensing  approaches  require  some  sort  of  light 
sensor  that  registers  what  the  eye  looks  like,  and  a 
microcomputer  that  computes  exactly  where  someone  is  gazing 
based  on  the  eye  image,  SST's  EyeTyper  model  100  uses  a  bulky 
video  camera  connected  to  a  powerful  microprocessor.  The 
EyeTyper  300  uses  an  inexpensive  and  compact  solid  state  light 
sensor  connected  to  the  same  microprocessor.  Over  several 
years,  SST's  personnel  have  developed  many  algorithms  that 
result  in  the  accurate  determination  of  gaze  position  given  the 
appearance  of  an  eye.  In  addition,  several  proprietary 
electronic  circuits  have  been  developed  which  greatly  assist 
the  cost  effective  incorporation  of  eye  gaze  sensing  technology 
into  products. 

The  low  cost,  helmet  mounted  eye  gaze  sensing  system  can  be 
broken  down  into  five  areas: 

1)  Eye  imaging. 

2)  Microprocessor  hardware 

3)  Real  time,  eye  image  processing  hardware. 

4)  Eye  image  processing  software. 

5)  Packaging. 

A  discussion  of  the  results  obtained  from  evaluation  of  the 
prototype  system  will  follow  this  description  of  each  area  as 
it  applies  to  the  working  phase  I  testbed. 
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EYE  IMAGING 


The  eye  is  composed  of  several  components:  the  iris  is  the  part 
that  determines  "what  color  your  eyes  are"  e.g.  blue  eyes  or 
brown  eyes.  It  also  expands  and  contracts  to  control  the  amount 
of  light  entering  the  eye  and  in  response  to  various  cognitive 
factors.  The  pupil  is  the  dark  area  that  is  surrounded  by  the 
iris.  Light  enters  the  eye  through  this  hole.  The  sclera  is  the 
white  area  that  surrounds  the  iris.  In  addition,  the  front  of 
the  eye  is  covered  by  a  transparent  surface  called  the  cornea. 


As  a  person  looks  around,  their  pupil  and  attached  cornea  move 
vertically  and  horizontally.  If  a  point  light  source  is 
positioned  in  the  field  of  view  of  the  eye,  it  will  be 
reflected  back  very  brightly  from  the  front  surface  of  the 
cornea.  This  is  because  the  cornea  partially  acts  as  a  convex 
reflector.  Hence,  as  the  eye  moves,  the  reflection  off  of  the 
cornea  will  move.  If  the  eye  gaze  sensor  is  fixed  relative  to 
the  user's  head,  or  if  supplementary  head  location/orientation 
information  is  available,  then  the  corneal  reflection  is 
sufficient  to  determine  the  point  of  gaze  (given  an  initial 
calibration).  An  alternative  technique  is  to  use  information 
about  the  pupil  location  along  with  the  corneal  reflection.  In 
a  helmet  mounted  system,  this  provides  more  tolerance  to  helmet 
movement  relative  to  the  user's  head  than  using  corneal 
reflection  alone.  As  a  person's  gaze  moves,  the  reflection  off 
of  the  cornea  relative  to  the  position  of  the  pupil  changes. 

This  is  the  eye  gaze  sensing  method  used  in  the  EyeTyper  model 
100.  This  technique  requires  more  computation. 

The  first  issue  to  be  dealt  with  when  building  a  helmet  mounted 
system  based  on  SST's  EyeTyper  technology  is  obtaining  an  image 
of  the  eye  within  the  helmet.  Three  specifications  greatly 
impact  this  part  of  the  design.  The  weight  added  to  the  helmet 
must  be  low,  the  production  quantity  price  of  the  system  must 
be  low,  and  the  helmet  mounted  packaging  requirements  must  be 
practical.  During  phase  I,  SST  personnel  searched  for  an 
imaging  technology  that  would  satisfy  these  constraints.  SST's 
early  products  (1984)  that  incorporated  eye  gaze  sensing 
(EyeTyper  100)  used  a  tube  based  video  camera.  While  this 
technology  supplies  a  good  quality  video  signal,  it  lacks 
compact  size,  robustness  to  mechanical  shock,  and  requires  a 
relatively  large  amount  of  power. 

With  the  recent  large  production  volumes  of  low  cost,  consumer 
video  products  (camcorders),  the  functional  performance  and 
price  of  CCD  (charge  coupled  device),  solid  state  image  sensors 
have  been  dramatically  improved  over  the  early  generation  CCD 
devices.  After  an  extensive  search,  SST  selected  a  lightweight, 
compact,  high  resolution  CCD  video  module  to  acquire  the  eye 
image.  This  video  module  is  made  by  Amperex  electronics  and  is 
composed  of  the  Phi lips/Amperex  NXA1031  CCD  image  sensor  (610  x 
488  pixels)  and  three  small  circuit  boards  attached  to  each 
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other  with  flex  circuit  connectors.  The  flex  connectors  enable 
the  CCD  video  module  to  be  mechanically  shaped  in  many 
different  ways.  This  is  tremendously  advantageous  for  the 
helmet  mounted  system  because  it  will  ultimately  enable  the 
video  module  to  be  packaged  conformal  to  the  side  of  the 
helmet.  The  module  can  be  conformed  to  a  4"  length,  2"  height, 
and  1"  depth  exclusive  of  imaging  optics  and  weighs 
approximately  8  ounces.  The  current  production  quantity  price 
of  the  video  module  is  $485.  It  outputs  a  NTSC  standard  video 
signal  (60  fields/sec)  and  consumes  minimal  power  (165  mA  at 
12V)  . 

To  obtain  the  corneal  reflection  and  to  illuminate  the  eye 
area,  low  power  (less  than  1/100  the  maximum  power  allowed  uder 
the  American  Conference  of  Governmental  Industrial  Hygienists 
guidelines  for  eye  infrared  exposure),  near-infrared  (850  nm) 
light  emitting  diodes  were  selected.  Because  infrared  light  is 
not  visible  by  humans,  it  does  not  interfere  with  the  user's 
visual  performance.  Silicon  based  CCD  arrays  are  very 
sensitive  to  the  near  visible,  IR  wavelengths  which  are  used. 

To  minimize  ambient  lighting  effects,  a  very  tight  IR  bandpass 
optical  filter  was  used  in  front  of  the  image  sensor.  This 
approached  has  worked  very  well  in  the  phase  I  system.  The 
near-lR  illumination  energy  is  carefully  channelled  so  that 
there  would  be  no  interference  if  used  in  conjunction  with 
night  vision  equipment.  The  physical  placement  of  the  camera 
and  light  sources  will  be  extensively  discussed  in  a  subsequent 
section  of  this  report. 

MICROPROCESSOR  HARDWARE 

The  microprocessor  hardware  must  be  able  to  support  both  the 
low  level  image  processing  software  algorithms  and  higher  level 
eye  gaze  mapping  software  algorithms.  In  addition,  it  must 
provide  for  sending  the  eye  gaze  vector  data  to  a  host  human 
factors  computer  or  flight  simulator.  During  phase  I,  SST 
constructed  a  "wire-wrapped"  prototype  circuit  board  that 
contained  an  8  bit,  65C02  microprocessor  running  at  2  MHz, 
static  RAM,  EPROM,  an  interface  to  the  eye  image  processing 
electronics,  and  a  serial  communications  port.  This  basic 
design  has  been  used  in  SST's  EyeTyper  products  and  supplied 
adequate  processing  power  for  the  purposes  of  the  phase  I 
testbed.  If  this  system  is  fully  developed  in  a  phase  II 
effort,  a  16  bit  68HC000  microprocessor  will  be  used  leading  to 
a  performance  improvement  with  no  significant  cost  penalty. 

This  issue  will*  also  be  discussed  in  a  subsequent  section  of 
this  report. 

EYE  IMAGE  PROCESSING  ELECTRONICS 

The  "wire-wrapped"  prototype  circuit  board  constructed  for  the 
phase  I  system  also  contained  circuitry  for  interfacing  the 
helmet  mounted,  CCD  video  camera  to  the  microprocessor.  This 
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proprietary  eye  image  processing  hardware  pre-processes  the 
digitized  video  image  from  a  CCD  camera  so  that  the  software 
can  quickly  determine  where  a  person  is  looking. 

It  consists  of  about  30  SSI,  MSI,  and  LSI  low  cost,  integrated 
circuits.  This  hardware  generates  a  2  level  (dark  and  bright) 
pseudo  run  length  encoded  representation  of  the  image.  The  two 
threshold  levels  are  totally  dynamic  -  one  is  used  for  the 
bright  corneal  reflection  and  one  is  used  for  the  dark  pupil. 

The  X  and  y  locations  of  threshold  level  crossings  are  stored 
in  a  512  x  18  FIFO  memory.  We  are  essentially  generating  a  list 
of  the  locations  of  dark  and  bright  edges  occuring  in  the  input 
image.  The  microprocessor  can  then  read  the  numbers 
representing  the  location  of  the  edges  from  the  FIFO  memory. 

This  enables  a  low  cost  system  to  process  eye  images  in  real 
time . 

This  basic  FIFO  memory-based  design  is  used  in  the  EyeTyper  300 
although  the  EyeTyper  300  uses  only  a  single  threshold  level. 
During  phase  I,  the  second  programmable  threshold  level  was 
added  to  the  basic  design  so  that  pupil  diameter  could  be 
computed  and  gaze  location  would  be  more  robust  to  the  helmet 
shifting  on  the  user's  head. 


EYE  IMAGE  PROCESSING  SOFTWARE 

65C02  assembly  language  software  running  on  the  microprocessor 
analyzes  the  run  length  encoded  image  data  to  determine  the 
location  of  the  user's  gaze.  The  software  initially  performs  a 
region  connectivity  analysis  of  the  edge  list  data,  thus 
generating  a  region  descriptor  list.  The  region  descriptor  data 
structure  contains  information  on  the  shape,  size,  and  location 
of  each  region.  After  this,  the  regions  representing  the 
corneal  reflection  and  the  pupil  are  chosen  from  the  region 
descriptor  list.  This  selection  process  is  straightforward 
because  the  pupil  and  corneal  reflection  regions  have  very 
distinguishing  characteristics  (unique  shape,  size  range, 
relative  location  between  pupil  and  cornea).  The  center  of  the 
corneal  reflection  and  the  center  of  the  pupil  are  then 
computed  (pupil  diameter  information  is  also  computed).  The 
vector  composed  of  the  x  and  y  differences  of  these  two  centers 
is  then  correlated  to  a  gaze  location  using  information 
obta’  .Ted  from  an  initial  calibration. 

As  stated  previously,  for  some  applications,  only  the  corneal 
reflection  may  be  required.  If  the  helmet  does  not  move 
relative  to  the  user's  eye  then  it  is  sufficient  to  determine 
gaze  location.  However,  if  the  pupil  and  corneal  reflection  are 
used  then  substantial  robustness  to  helmet  shifting  is 
obtained.  This  has  been  demonstrated  on  the  phase  I  system. 
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For  the  phase  I  testbed,  once  the  x-y  gaze  vector  is  computed, 
it  is  sent  to  a  host  SUN  Microsystems  workstation  via  a  4800 
baud,  RS-232  link.  If  no  eye  is  seen  by  the  system  then  it 
sends  a  null  (all  zero)  gaze  vector.  This  allows  monitoring  of 
eye  blinks.  Software  written  in  the  C  programming  language 
running  on  the  SUN  computer  allowed  for  experimentation  and 
demonstration  of  helmet  mounted  eye  gaze  sensing. 

To  accomodate  anatomical  variability  between  users  and  the 
variability  of  eye  location  within  the  helmet  relative  to  the 
eye  imaging  environment,  eye  tracking  systems  require  the  user 
to  go  through  a  calibration/training  process.  It  is  most 
desirable  to  make  this  as  quick  and  robust  as  possible. 

Software  on  the  SUN  microsystems  computer  implemented  a  three 
point  gaze  calibration  sequence.  After  this  calibration  was 
completed,  the  area  on  the  SUN  bitmap  that  the  user  was  looking 
at  was  indicated  with  a  dark  square. 

PACKAGING 

Because  of  limited  funding,  phase  I  effort  was  designed  to 
prove  technical  feasibility,  not  produce  a  production  quality 
product,  the  packaging  of  the  system  was  not  a  primary  concern. 
However,  it  was  very  important  that  the  overall  conceptual 
design  be  amenable  to  a  robust,  practical  package  given  full 
development  in  a  phase  II  effort.  For  this  reason,  an  extensive 
search  was  performed  to  find  an  imaging  device  which  would  meet 
the  specifications.  The  chosen  Amperex,  CCD  camera  is  compact, 
lightweight,  and  its  flexible  circuit  board  construction  will 
greatly  facilitate  the  helmet  mounted  packaging  of  a  phase  II 
system. 

The  phase  I  testbed  uses  a  mechanical  linkage  attached  to  an 
oxygen  mask  latchpoint  on  the  helmet  to  facilitate 
experimentation  with  the  position  of  the  lightweight,  CCD 
camera  relative  to  the  user's  eye.  Results  of  these  experiments 
have  led  to  the  packaging  design  for  a  production  quality 
device  as  described  in  the  phase  II  proposal  and  summarized  at 
the  end  of  this  report.  The  bulky  mechanical  linkage  of  the 
phase  I  device  will  not  be  needed. 

The  helmet  mounted,  compact,  CCD  camera  is  connected  via  a 
cable  to  a  7"  x  11"  prototype,  "wire-wrapped"  circuit  board 
containing  the  eye  image  processing  electronics.  This  circuit 
board  is  housed  in  a  metal  case  which  provides  for  connection 
of  the  CCD  camera,  a  power  supply,  video  monitor  to  view  the 
unprocessed  eye  image,  video  monitor  to  view  the  processed  eye 
image,  and  a  DB-25  connector  for  the  serial  digital 
communications  link. 
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Summary  of  Phase  I  Low  Cost,  Helmet  Mounted  Eye  Gaze  Sensing  Testbed 

During  phase  I,  Sentient  Systems  Technology  (SST)  has 
demonstrated  to  Navy  personnel  that  application  of  its  low 
cost,  eye  gaze  sensing  technology  will  result  in  a  practical, 
low-cost,  helmet  mounted  eye  gaze  sensing  system  for  use 
throughout  the  Department  of  Defense.  SST  has  constructed,  and 
demonstrated  to  Navy  personnel,  a  low  cost,  helmet  mounted,  eye 
gaze  sensing  testbed.  NATC  personnel  have  operated  the  working 
phase  I  system  and  were  extremely  pleased  with  its  operation.  A 
photograph  of  the  helmet  mounted  section  of  the  phase  I  system 
showing  the  image  of  the  eye  on  a  video  monitor  is  included  at 
the  end  of  this  section. 

This  testbed  consists  of  a  low  weight  (8  ounces)  CCD  camera 
mounted  to  a  Navy  flight  helmet  via  a  mechanical  linkage  which 
facilitates  experimentation  with  various  camera  angles  for 
viewing  the  pilot's  eye.  The  camera  is  connected  to  a  "wire 
wrapped"  circuit  board  which  contains  a  microprocessor  and 
custom,  eye  image  processing  electronics.  Several  near-lR  light 
emitting  diodes  can  be  positioned  for  experimentation  with 
different  eye  illumination  techniques.  Software  running  on  the 
microprocessor  analyzes  images  of  the  eye  and  outputs  an  x-y 
gaze  vector  to  its  RS-232  serial  port.  The  eye  image  processing 
circuit  board  is  connected  to  a  SUN  Microsystems  workstation 
via  a  RS-232  serial  link. 

A  user  can  sit  in  front  of  the  workstation  wearing  the  helmet 
and  control  a  cursor  on  the  bitmap  screen  by  shifting  gazepoint, 
i.e.,  the  area  on  the  screen  the  user  is  looking  at  is  highlighted 
with  a  black  square.  The  user  goes  through  a  three  point 
calibration  sequence,  then  software  on  the  SUN  computer  draws  a 
grid  of  the  appropriate  resolution,  updating  the  user's  gaze 
point  in  real  time  (approximately  45  times  a  second).  Gaze 
resolution  to  approximately  1/2  a  degree  has  been  demonstrated. 
Because  the  x-y  gaze  vectors  input  to  the  SUN  computer  are 
relative  to  the  helmet  frame  of  reference,  the  user's  head  must 
remain  fixed  relative  to  the  SUN  display  in  order  for  the 
indicated  point  of  gaze  to  be  correct.  If  a  person  blinks  their 
eyes,  then  no  gaze  position  is  indicated  on  the  screen.  In  this 
way,  blinks  are  easily  monitored.  Pupil  diameter  can  also  be 
computed  by  the  microprocessor  software.  There  is  evidence  that 
this  may  be  an  indicator  of  workload  stress. 
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Evaluation  of  the  Phase  I  Testbed 

The  above  described  system  constructed  during  phase  I  has 
satisfied  the  objectives  defined  for  this  limited  funding, 
technical  feasibility  demonstration  project.  It  has  proven  the 
technical  feasibility  of  the  approach,  allowed  prediction  of 
operational  parameters  for  the  fully  developed  device,  and  has 
shown  the  engineering  design  space  that  will  be  optimized 
during  the  phase  II  development.  We  will  now  describe  areas 
that  will  need  to  be  more  fully  developed  in  a  phase  II  effort 
based  on  the  evaluation  of  the  phase  I  testbed  and  conclude 
with  a  discussion  of  a  helmet  eye  imaging  study  performed 
during  phase  I  with  the  testbed. 

MICROPROCESSOR 

The  phase  I  testbed  uses  an  6  bit,  2  MHz,  65C02  microprocessor. 
While  sufficient  for  the  phase  I  system,  substantially  higher 
performance  levels  will  be  attained  by  using  a  more  powerful  16 
bit,  12.5  MHz,  66HC000  microprocessor  for  the  proposed  phase  II 
system.  All  computer-based  devices  can  benefit  from  the 
availability  of  more  compute  cycles  and  the  low  cost,  helmet 
mounted,  eye  gaze  sensing  system  is  no  exception.  Because  this 
increase  in  computing  power  can  be  obtained  with  no  cost 
penalty,  it  is  desirable  to  incorporate  it  into  the  phase  II 
system. 

The  theoretical  maximum  speed  attainable  using  low  cost,  off 
the  shelf,  CCD  camera  technology,  set  by  the  NTSC 
synchronization  standard,  is  60  eye  gaze  vectors  per  second. 

The  actual  speed  of  an  operating  system  is  limited  by  the  point 
of  gaze  computation  time  required  by  the  software  operating  on 
the  host  microprocessor  in  conjunction  with  the  custom  eye 
image  processing  hardware.  The  phase  I  testbed,  using  the  8  bit 
microprocessor,  operates  at  about  45  gaze  vectors  per  second 
while  doing  corneal  tracking  only.  Combination  pupil-cornea 
tracking  is  slightly  slower  (40  per  second).  A  faster 
microprocessor,  running  the  same  software  algorithms,  will 
increase  the  gaze  sensing  speed  of  the  system  and  should 
approach  the  theoretical  maximum  rate. 

In  addition  to  increasing  the  speed  of  the  system,  a  more 
powerful  microprocessor  will  allow  the  use  of  increasingly 
sophisticated  software  algorithms.  For  example,  the  phase  I 
system  computes  the  center  of  the  pupil  by  using  a  center  of 
mass  approach.  This  works  well  if  the  pupil  region  is  imaged 
such  that  it  has  a  regular  shape,  but  does  not  work  as  well  if 
the  pupil  region  becomes  connected  with  other  dark  eye  features 
such  as  the  top  eyelid.  This  can  partially  be  dealt  with  by 
using  a  lower  camera  angle,  however  the  best  solution  is  to  use 
a  more  sophisticated  center-of-pupil  algorithm.  Computer 
vision,  region  shape  analysis  algorithms  can  result  in  an 
accurate  measure  of  the  center  of  the  pupil  and  its  diameter 
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given  irregularly  imaged  pupils  but  require  more  computation 
time  than  simpler  approaches. 

CALIBRATION  and  GAZE  MAPPING  SOFTWARE 

Once  the  calibration  information  has  been  gathered  by  the 
system,  it  must  be  used  to  map  eye  features  to  their  proper 
gaze  vector.  The  phase  I  system  has  shown  that  while  a 
straightforward  linear  mapping  will  work  at  lower  resolutions, 
because  the  cornea  is  spherical  at  the  center  and  flattens  out 
toward  its  edges,  nonlinear,  second  order  fitting  functions 
must  be  used  to  obtain  the  best  accuracy  at  the  highest  gaze 
resolution.  This  also  necessitates  the  use  of  a  more  powerful 
microprocessor  than  that  used  in  the  phase  I  testbed. 

The  phase  I  system  uses  a  three  point  calibration  sequence  for 
which  pacing  and  presentation  is  automatic.  The  user  needs 
only  to  look  at  the  given  gazepoints  and  therefore  this  process 
takes  approximately  10  seconds  to  complete.  This  has  resulted 
in  excellent  performance  at  coarser  resolutions  but  a  small 
number  of  additional  calibration  points  will  be  needed  for 
maximum  accuracy  at  the  highest  resolution.  Extensive  testing 
and  experimentation  will  be  done  on  this  calibration  mapping 
problem  during  phase  II. 

HELMET  MOUNTED  EYE  IMAGING 

Because  the  helmet  mounted  system  must  be  usable  with  visual 
enhancement  devices  (e.g.  night  vision)  and  the  visual 
performance  of  the  pilot  must  not  be  degraded,  an  oblique 
camera  angle  view  of  the  eye  is  desirable.  The  initial  phase  I 
system  used  a  helmet  mounted  camera  angle  of  approximately  35 
degrees  temporal  to  the  eye  and  a  single  near-IR  light  emitting 
diode  positioned  at  this  same  angle  to  obtain  the  corneal 
reflection.  With  this  single  light  source,  single  oblique 
camera  arrangement,  the  reflection  of  the  light  source  off  of 
the  cornea  of  the  eye  was  observable  for  all  practical  temporal 
gaze  angles  but  only  observable  up  to  gaze  angles  5  degrees 
nasal  to  the  midline. 

Further  use  of  the  testbed  system  indicated  to  SST  personnel 
that  an  extensive  helmet  mounted,  eye  imaging  design  study 
should  be  done.  It  appeared  to  SST  personnel  that  an  extremely 
wide  range  of  eye  gaze  angles  could  ultimately  be  accomodated 
in  a  fully  developed  system  that  used  multiple  light  sources 
and  two  cameras  each  viewing  an  eye  from  an  oblique  angle.  The 
fundamental  issue  to  be  investigated  was  determination  of  the 
range  of  resolvable  gaze  angles  given  a  camera  viewing  angle 
and  a  position  for  the  light  source  causing  the  corneal 
reflection . 
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A  helmet  mounted  eye  imaging  experimental  system  was 
constructed  that  enabled  the  camera  angle  of  view  to  be  varied, 
the  position  of  a  point  light  source  to  be  varied,  and  the 
resulting  resolvable  gaze  angle  to  be  measured.  This  consisted 
of  the  lightweight,  CCD  camera  used  in  the  operational  phase  I 
system,  an  American  Optical  Company  field  of  view  measurement 
instrument,  and  a  near-IR  LED  on  a  flexible  arm  for  positioning 
at  various  3-D  locations. 

EYE  GAZE  FIELD  DETERMINATION  CHARTS 

For  a  given  camera  viewing  angle  and  a  given  light  source 
position,  the  field  of  view  in  which  the  corneal  reflection  was 
usable  for  gaze  sensing  purposes  was  plotted  on  an  American 
Optical  Company  field  of  view  chart.  The  coordinate  system  is 
eye-centered  not  helmet-centered;  the  data  presented  is  from  a 
user's  left  eye.  Symmetrical  results  are  obtained  from  the 
corresponding  right  eye  data. 

SYMBOL  KEY 

The  camera's  angular  position  in  this  coordinate  system  is 
indicated  on  each  chart  with  a  "C"  and  the  light  source  angular 
position  is  indicated  with  an  "L".  The  solid  line  on  the 
following  charts  represents  the  measurable  limit  of  gaze  angle 
obtainable  from  a  reflection  off  of  a  user's  cornea.  The  dotted 
line  represents  limits  imposed  by  physiological  constraints  or 
eyelid  obstruction. 
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canera  Is  located  approxljnately  15  degrees  below  the  horizontal  plane  and  70 
degrees  temporal  from  the  midline. 

Figure  as  reference  light  on  the  horizontal  plane,  70  degrees  tewporal . 

Figure  b:  reference  light  la  centered  on  the  left  eye  field  of  view. 

Figure  c:  reference  light  Is  45  degrees  nasal  and  20  degrees  below  the 
horizontal . 
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Ca»ra  is  located  approxivately  15  degrees  below  the  horlsontal  and  45  degrees 
teeftoral  £ron  the  mid  line. 

Figure  at  reference  light  near  the  horizontal  plane,  45  degrees  temtoral. 

Figure  bi  reference  light  45  degrees  laelow  horizontal,  50  degrees  ttngirinl 
Figure  ci  reference  light  is  centered  on  the  left  eye. 

Figure  di  reference  light  is  on  the  horizontal  and  45  degrees  nasal. 
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Caaeza  is  located  45  degrees  tewporal  and  approxiaately  35  degrees  below  the 
horizontal . 

Figure  a:  reference  light  20  degrees  below  horizontal  plane,  and  centered. 
Figure  b:  reference  light  15  degrees  below  horizontal,  45  degrees  tewporal. 
Figure  c:  reference  light  is  15  degrees  below  horizontal  and  45  degrees  nasal. 
Figure  d:  reference  light  is  centered  on  the  left  eye. 
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Helmet  Mounted,  Eye  Imaging  Design  Study  Summary 

In  figure  la  we  see  the  expected  broad  symmetrical  detectable 
gaze  pattern  from  the  centrally  located  camera,  in  contrast, 
figure  lb  shows  the  highly  asymmetric  and  irregular  detectable 
gaze  pattern  that  is  a  consequence  of  the  top  eyelid, 
anatomical  interference  with  the  reflection  from  the  raised 
light  source  which  is  above  the  central  camera  location.  In  the 
actual  experiment,  the  centrally  located  camera  obscured  a 
significant  fraction  of  the  subject's  central  field  of  view. 

In  a  practical  helmet-mounted  system,  the  camera  would  be 
physically  located  outside  the  user's  field  of  view,  and  both 
relay  and  combiner  optics  would  be  used  to  provide  this  vantage 
point  to  the  camera. 

Figure  2  shows  a  series  of  experiments  with  the  camera  located 
in  the  extreme  periphery  of  the  field  of  view.  With  the  light 
source  above  the  camera,  the  detectable  gaze  angles  are 
symetric  above  and  below  the  horizontal.  As  the  light  source 
is  moved  progressively  more  nasal,  the  same  shift  is  seen  in 
the  detectable  range  of  horizontal  gaze.  The  apparent  broad 
range  of  detectable  gaze  angles  seen  in  figures  2b  and  c  is 
misleading  since  the  gaze  angle  measurements  are  imprecise  in 
the  nasal  directions. 

In  figure  3,  we  show  the  camera  location  that  we  feel 
represents  the  best  compromise  between  a  desirable  peripheral 
camera  position  and  both  detectable  gaze  range  and  gaze 
resolution.  Figures  3a,  c,  and  d  show  various  positions  of  the 
light  source  in  a  horizontal  plane  at  the  vertical  zero.  As 
the  light  is  moved  from  45  degrees  temporal,  through  zero,  to 
45  degrees  nasal,  the  range  of  detectable  gaze  angles  increases 
in  the  nasal  direction.  From  figure  c  and  d,  it  may  be 
observed  that  if  the  light  source  is  too  far  nasally  located, 
detectable  gaze  range  and  resolution  on  the  side  of  the  camera 
is  degraded.  Similarly  (figure  3b),  positioning  the  light 
source  below  the  camera  seriously  degrades  range  of  gaze 
detectibility . 

It  might  be  supposed  that  by  further  lowering  the  camera 
position,  there  will  be  less  interference  with  the  corneal 
reflection  by  the  upper  eyelid,  and  thus  the  range  of 
detectable  eye  gaze  angles  should  also  increase.  Figure  4 
demonstrates  that  this  is  not  the  case  for  either  the  vertical 
or  horizontal  angles. 

From  the  foregoing  demonstrations,  we  may  conclude  that  a 
reliable,  helmet-mounted,  eye  gaze  direction  detection  system 
can  be  built  with  two  laterally  mounted  cameras  and  three  visor 
mounted,  near-infrared,  light  sources.  Figure  3  demonstrated 
that  a  45  degree  temporal  camera  position  provided  for  adequate 
range  and  resolution  of  detectable  gaze  angle  when  only  two 
light  source  locations  are  used.  This  vantage  point  may  be 
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achieved  when  a  camera  is  positioned,  completely  out  of  the 
pilot's  field  of  view  (near  the  oxygen  mask  latchpoint),  with  a 
simple  dichroic  beamsplitter.  This  optical  design  will  not 
require  complex  relay  elements  and  should  be  virtually 
transparent  to  the  user.  From  this  45  degree,  temporal  camera 
vantage  point,  a  single  temporal  light  source  and  another  light 
source  mounted  on  the  visor  midline  (resulting  in  it  being 
significantly  nasal  to  each  eye),  will  permit  a  one  camera 
system  to  crudely  track  the  full  range  of  gaze  motions  and  a 
two  camera  system  to  track  the  full  range  at  full  resolution. 
The  dichroic  beamsplitter  will  be  located  at  45  degrees 
temporal  on  the  horizontal  arc  and  thus  will  be  off  the 
visual  gaze  line  for  most  look  angles. 


Predicted  Cost 

As  stated  previously,  the  deliverable  production  cost  of  this 
helmet  mounted,  eye  gaze  sensing  system  is  one  of  the  primary 
motivating  factors  for  this  effort.  The  phase  I  testbed  has 
shown  that  the  production  quantity  cost  of  a  dual  camera 
system,  if  fully  developed  in  a  phase  II  effort,  will  include: 
1)  2  lightweight,  CCD  cameras  at  $485  per  camera;  2)  eye  image 
processing  printed  circuit  boards  at  a  cost  of  approximately 
$800;  3)  helmet  mounted  optics  and  helmet  mounted  packaging  at 
an  approximate  cost  of  $1,000;  4)  power  supply  and  packaging 
for  eye  image  processing  electronics  at  approximately  $100;  and 
5)  assembly  and  testing  labor  at  ~$300.  This  should  allow  room 
for  any  unforeseen  costs  and  a  fair  commercial  profit  margin 
while  still  meeting  the  under  $10,000  deliverable  production 
cost  specification. 


Phase  II  System  Design 

Based  on  the  evaluation  of  the  phase  I  demonstration  testbed,  a 
conceptual  design  for  a  production  quality,  low  cost,  helmet 
mounted,  eye  gaze  sensing  system  has  been  generated  and 
described,  in  detail,  in  a  phase  II  proposal.  Briefly,  it 
includes:  1)  using  a  16  bit  68HC000  microprocessor  for 
increased  performance;  2)  conformally  side  mounting  two  CCD 
cameras  near  the  oxygen  mask  latchpoints  such  that  they  do  not 
intrude  into  the  pilot’s  field  of  view;  3)  unobtrusive  dichroic 
mirror  optics  to  direct  the  view  of  the  eye  to  the  CCD  cameras; 
4)  multiple,  small,  near-IR,  point  light  sources  to  obtain  wide 
gaze  angle  capability;  and,  5)  a  highly  flexible,  low  mass 
cable  connecting  the  two  helmet  mounted  camera/IR  control 
modules  with  the  remotely  mounted  image  processing  hardware. 

The  remote  hardware  module  and  helmet  module  are  separate, 
interchangeable  entities,  i.e.,  a  user  could  purchase  several 
helmet  modules  (small,  medium,  large)  and  only  one  eye  image 
processing  hardware  module.  The  system  digitally  outputs  x-y 
gaze  location,  pupil  diameter,  and  a  timestamp. 
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